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Abstract 



We present a simple randomized algorithm that approximates the number of satisfying assign- 
ments of Boolean formulas in conjunctive normal form. To the best of our knowledge this is the first 
algorithm which approximates #fc-SAT for any k > 3 within a running time that is not only non- 
trivial, but also significantly better than that of the currently fastest exact algorithms for the problem. 
More precisely, our algorithm is a randomized approximation scheme whose running time depends 
polynomially on the error tolerance and is mildly exponential in the number n of variables of the 
U^ ■ input formula. For example, even stipulating sub-exponentially small error tolerance, the number of 

c/5 ' solutions to 3-CNF input formulas can be approximated in time 0(1.5366"). For 4-CNF input the 

O. bound increases to 0(1. 6155"). 

We further show how to obtain upper and lower bounds on the number of solutions to a CNF 
formula in a controllable way. Relaxing the requirements on the quality of the approximation, on 
fc-CNF input we obtain significantly reduced running times in comparison to the above bounds. 

o 

^ ! 1 Introduction 

The design and analysis of algorithms that determine the satisfiability or count the models of fc-CNF 
formulas has quite some tradition. In the case of the satisfiability problem the earliest algorithm with a 
worst case running time which is significantly better than the trivial poly(n)2 n bound dates back to at 
least 1985 HMS851 . The time bounds have improved gradually over the years with most recent results 
k> | (only a few of which are KT041 IR0IO6I IHMS111 IHerlllO being analyses of randomized algorithms 

that have been obtained from either Scheming 's algorithm ||Sch99L the algorithm of Paturi, Pudlak, 
Saks, and Zane HPPSZ051 . or a combination of both. The currently fastest algorithm for 3-SAT by 
Hertli BHerllll running in time 0(1.30704") falls roughly into the second category. The corresponding 
counting problems have seen similar improvements MDub911IZha961lDJW051lKut07IIFK071IWa h081 over 
the trivial time bound, with the current best worst case running time for #3-SAT being 0(1.6423™ ) 
obtained by Kutzkov IKut07l . 

Quite surprisingly, however, the situation is completely different for the approximation of #k-SAT. 
To the best of my knowledge not even small improvements over the trivial worst case time bound are 
knownlU This, however, does not seem to be due to a general lack of interest in the problem itself. 
From a complexity theoretical point of view, for example, several already classic papers BSto761 IVV861 
IRot961 study questions closely related to direct algorithmic problems in #/c-SAT approximation. In 
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particular Valiant and Vazirani BVV86II bound the complexity of the approximation problem from above 
by reduction to SAT, and hence settle its complexity in a certain sense. 

While theoretical results on the approximation of #fc-SAT are rather old, there are several heuristic 
approaches to the problem, that have all appeared only fairly recently. Motivated by questions of prac- 
ticability, these results focus on methods that can be shown empirically to work well, while sacrificing 
some (at least theoretically) desirable properties. That is, some of these approaches yield approximations 
without any guarantee on the quality of the approximation MWS051IGD07II . Others yield reliable lower 
and upper bounds BGSS06I IGHSS071 IKSSlll which, in certain cases, are surprisingly good although 
generally their quality is rather low. In particular, this line of work does not provide rigorous bounds on 
running times and neither does it yield rigorous quality estimates of the approximation computed. 

With regard to the above results, the lack of competitive worst case bounds for #fc-SAT approxima- 
tion algorithms seems to be due to several factors. First of all the exact algorithms found in the literature 
and their analyses do not seem to carry over easily to the approximation problem. Secondly, complexity 
theoretical insights are usually not considered applicable in the context of designing fast exponential 
algorithms. An example is the technique of Valiant and Vazirani which leads to a significant blow up 
in formula size. And thirdly, it is not clear which of the known algorithmic ideas used in the heuristic 
approaches could at least in principle show a good worst case behavior. 

1.1 Contributions 

In this paper we will see that one can indeed not only improve upon the trivial worst-case time bound 
mentioned above. But the algorithm we will present also provides arbitrarily good precision in signifi- 
cantly less time than known exact methods. To be more precise, the algorithm we present is a randomized 
approximation scheme for #fc-SAT for every k > 3. Given a freely adjustable error tolerance e > 0, 
randomized approximation schemes produce an output that is within a factor of e e of the number #F of 
solutions of some input formula F. 

We obtain the following main result, which we state here only for k = 3 and k = 4. The general 
result will be discussed in Section [3] 

Theorem 1.1. There is a randomized approximation scheme running in time 0(e~ 2 ■ 1.5366 71 ) for #3- 
SATand in time 0(e" 2 • 1.6155 n )/or #4-SAT. 

For #3-SAT this algorithm is already significantly faster than the currently fastest exact algorithm 
from MKut071 which runs in time 0(1.6423 n ). For ^4-SAT the benefit of approximation is even more 
impressive, as the best bound for exact methods is still the 0(1.9275") bound of the basically identical 
algorithms of Dubois Pub91| and Zhang HZha96l 

We will see that the algorithm of Theorem [TTT] is not complicated and monolithic like the branching 
algorithms usually employed in exact counting results. But it is actually a combination of two very 
simple and very different algorithms. The main reason for considering this combination relies on two 
pieces of intuition. On the one hand, if a formula has few solutions, then it is not too bad an idea to 
compute their number by simply enumerating them. On the other hand, if a formula has many solutions, 
then a quite trivial sampling algorithm should yield good results. 

Observe that the result of Theorem 11.11 can already be used to compute e.g. j^F exactly in time 
0(1.5366") for any 3-CNF formula which has only a sub-exponential number of solutions. To achieve 
this we only have to set e appropriately. However, we shall see below, that this can also be achieved in 
significantly less time. Motivated by the heuristic results on the approximation of #fc-SAT described 
above, we also study the effect of weaker requirements on the approximation bounds. It seems, of 
course, perfectly reasonable to assume that weaker bounds should come at the benefit of dramatically 



improved running time bounds. We will therefore show that this is the case. With respect to lower 
bounds we obtain: 

Theorem 1.2 (Lower Bound Algorithm). There is a randomized algorithm which, on input a 3-CNF 
formula F on n variables and a natural number L, performs the following in time 0(L 0,614 ■ 1.30704"); 

• IfjfF > L it reports this with probability at least 3/4. 

• If#F < L then with probability at least 3/4 it reports this and outputs the correct value i^F. 
Furthermore, there is a deterministic algorithm solving this task in time O(iV ' 585 ■ 1.3334"). 

This lower bound algorithm will in fact be used in the proof of Theorem 11.11 and relies on the 
above observation that we can simply use a SAT algorithm for enumerating all solutions provided the 
input formula has only few. The time bounds mentioned thus arise from the SAT algorithms used - the 
randomized 3-S AT algorithm by Hertli IIHerl 111 and the deterministic one of Moser and Scheder HMSlll 
(which is in fact a derandomized version of Schoning's algorithm). 

To obtain upper bounds, on the other hand, we cannot use the high-solution part of Theorem 11.11 
But, although it might seem unreasonable to expect that this would yield a competitive running time, 
we can use an algorithm based on the bisection technique of Valiant and Vazirani [VV86|. Interestingly 
an algorithm based on Valiant and Vazirani 's technique has been used already in the heuristic result 
of BGSS06II . Their approach, however, is quite different from ours and does not have a good worst-case 
behavior. 

By systematically augmenting and input formula F with randomly chosen GF(2)-linear constraints, 
the bisection technique makes it possible to approximate j^F by determining satisfiability of the aug- 
mented formulas. The main difference of our approach to this classical scheme lies in the observation 
that it is more reasonable for our purposes to work directly with the system of linear equations obtained, 
instead of encoding it into fc-CNF. In this way we obtain the running time bounds which are valid even 
for general CNF input formulas. 

Theorem 1.3 (Upper Bound Algorithm). There is an algorithm, which on input a CNF formula F on 
n variables and an integer [i < n takes time 0*(2 n ~^) and performs the following with probability at 
least 2/3: 

It outputs a number u > p such that U : = 2 M+3 > #F. If furthermore u > p then 2 U is a 
16-approximation ofj^F. 

Remark. This algorithm will actually work for upper bounding \S\ for any set S C {0, 1}" with a 
polynomial membership test. However, as this is a trivial consequence of the proof, we consider only 
the case that the input is a CNF formula. 

Moreover, we do not particularly focus on improving the approximation ratio mentioned in The- 
orem 11.31 Such an improvement is in fact unnecessary if we want to use this algorithm to design a 
randomized approximation scheme: We can combine the above algorithm with that of Theorem [L2] to 
obtain a 16-approximation algorithm for e.g. #3-SAT which runs (up to a polynomial factor) within in 
the same time bound as that stated in Theorem 11.11 This algorithm can then be plugged into a Markov 
chain by Jerrum and Sinclair HSJ891 to boost the quality of approximation. This yields a (1 + A >, )- 
approximation algorithm incurring only a polynomial overhead in the computation. Thus we have a 
second, although more complicated, algorithm that satisfies the claim of Theorem ll.il 



2 Preliminaries 

For a CNF formula F, let sat(F) be the set of its solutions and #F = |sat(F) |. We shall always use n to 
denote the number of variables of a CNF formula under consideration. A randomized a-approximation 
algorithm A for #/c-SAT outputs, on input a fc-CNF formula F, a number A(F) such that 

Pr [a _1 #F < A(F) < a#F] > p. (1) 

Where p is some constant, independent of the input and strictly larger than_| 1/2. A randomized approx- 
imation scheme for #A;-SAT, is then an algorithm which on input F and a natural number e~ x behaves 
like a randomized e e -approximation algorithm. 

We use the notation x 1 = x and x° = x. For a clause C, a variable x, and a truth value a G {0, 1}, 
the restriction of C on x = a is the constant 1 if the literal x a belongs to C, and C \ {x l ~ a } otherwise. 
We write C\ x=a for the restriction of C on x = a. A partial assignment is a sequence of assignments 
(x\ = ai, . . . ,x r = a r ) with all variables distinct. Let a be a partial assignment. We will use the 
notation a U (x = a) to denote the assignment (x± = oi, . . . , x r = a r , x = a). If C is a clause, we let 
C | a be the result of applying the restrictions x\ = a\ , . . . , x r = a r toC. Clearly the order of application 
does not matter. If F is a CNF formula, we let F\ a denote the result of applying the restriction a to each 
clause in F, and removing the resulting l's. We call F\ a the residual formula. 

As we will use the algorithm of Paturi, Pudlak, Saks, and Zane MPPSZ051 and a very recent paper by 
Hertli HHerllll . we need the constant 

Vk = ^ ■ 



t M + 



fc-i- 



3 The Algorithm 



We are now able to state the main result in full detail. 

Theorem 3.1. For k > 3, #k-SAT has a randomized approximation scheme running in timfQ 



O* e~ z -2 U-i+M fc , 



As already outlined, the randomized approximation scheme of Theorem l3.1l is a combination of two 
different algorithms. We will discuss the algorithm for the case of few solutions now. The case of many 
solutions will be treated afterwards in Section [3^21 

3.1 Formulas with few solutions 

For formulas with few solutions we will now present an algorithm relying on a simple enumeration of 
solutions by using a /c-SAT algorithm as a subroutine. This will also prove Theorem 1 1.21 

Lemma 3.2. Let F be a k-CNF formula on n variables and let A be an algorithm performing the 
following task in time 0*{2" kTl ). If F is satisfiable, with probability at least 3/4 it outputs a solution to 
F.IfF is unsatisfiable, it reports this correctly. 

Then, there is a algorithm, which on input F and a natural number N , takes time 0*(2" kn N^ 1 ~" k '), 
and performs the following: 



"In the literature, usually either the value p — 3/4 or a further parameter S such that p = 1 — 5 seems to be common. 
However, it is well-known that all of these can be translated into each other with only polynomial overhead. 
3 We use the O* notation to suppress factors sub-exponential in n. 



• If#F > N it reports this with probability at least 3/4. 

• If#F < N then with probability at least 3/4 it reports this and outputs the correct value #F. 
Furthermore, if the algorithm reports #F > N then this holds with certainty. 

Theorem ll.2l follows directly from this lemma by using the randomized 3-SAT algorithm of Hertli HHerlll 
which has (3% = 0.3864. For the claim about the deterministic algorithm we use the result of Moser and 
Scheder IMS 111 , with fo = 0.4151. 

In the proof of the above lemma, we will use the following fact which is very easily proven. 

Lemma 3.3. A rooted tree with N leaves and depth (i.e. max root to leaf -distance) n has at most n ■ N 
vertices in total. 



Proof of Lemma \372\ Note first, that by a standard trick we can boost the success probability of A. 
Assume that, as provided by the statement of the lemma, we have error probability at most 1 — p < 1/4. 
Then the probability of erring in M independent repetitions is at most (1 — p) M < e~ pM . Call the 
boosted version of this algorithm A*. 

We shall fix a good value for M. Below we will see that algorithm A* will be queried a number 
0(nN) of times, for some N < 2", each time on a formula of at most n variables. The probability 
that the algorithm errs in any of these queries, is at most nN ■ e~ pAI . So choosing M within a constant 
factor of log nN (which is polynomial in n) allows us to condition on the SAT algorithm not erring in 
any of the 0(nN) queries. The probability of that latter event to happen is close to 1. And as this is the 
only possible source of failure of the algorithm, we will easily achieve a success probability of 3/4 in 
the end. 

The algorithm. Check if F is satisfiable, using A*, and if so, perform the following. Inductively, 
construct a search tree associated with partial assignments a, such that F\ a is satisfiable. For a leaf 
in the current search tree associated with some assignment a, choose a variable x from F\ a and check 
F\au(x=o) an d F\aLi(x=i) f° r satisfiability using the algorithm A*. For each of the satisfiable restrictions 
add a new child to the current leaf in the search tree. We stop the algorithm, if it has N leaves, or if it 
has found all of the solutions of F. Traversing this tree, e.g. in a depth first manner we can implement 
this procedure in polynomial space (not taking the space need of A* into account). 

Time. Consider the search tree this algorithm produces. As it has at most N leaves, and depth at most 
n, we have (recall Lemma 1331) at most nN nodes overall in the tree. 

Observe first, that at each node we perform at most 2 queries to the SAT algorithm A*. A node of 
level £ in the tree incurs queries taking time at most 0*(b n ~ e ) for b = 2" k . Therefore, we can give 
an upper bound of the overall time spent in answering all queries by bounding the time spend on a 
completely balanced binary search tree of depth d = log nN. 

Let T(d, n) denote the overall time spend to run the algorithm on a balanced binary search tree with 
d levels with an n variable formula. Then, up to a sub-exponential factor for the time spent at each node 
in the tree, T(d, n) = b n + 2T(d - 1, n - 1). Note that T(0, n) = 1, and thus 



T(d,n) = y }T2 u b n - u 



v=0 

which yields the claimed bound. □ 



3.2 Formulas with many Solutions. 

We use the following simple folklore algorithm which can be found e.g. in Motwani and Raghavan's 
book IIMR95I . Given a CNF formula F on n variables, choose an assignment from {0, l} n uniformly 
at random. Repeat this process a number N of times and let X be the number of solutions of F among 
these N trials. By a simple argument (see e.g. Theorem 11.1. in IIMR95I0 . if 

N = n~ 



nn 

then with probability at least 3/4, the value X • — is an e e -approximation of #F. Hence, we have the 



^#F 

probability at least 3/4, the value X • — 
following 

Lemma 3.4. Let F be an n variable CNF formula with at least N solutions and e" 1 a natural number. 

( 2 n \ 
Then there is an algorithm which, in time O* ( ) yields a randomized e e -approximation of#F. 



e 2 N 



3.3 Combining the algorithms 



We shall now prove Theorem l3.1l by combining both of the above algorithms. Let F be a /c-CNF formula 
on n variables and e~ x a natural number. We run the algorithm of Lemma [3T21 with a parameter N. The 
exact value of N will be determined later. Note that if the algorithm reports #F < N, it also computes 
#F exactly with probability at least 3/4. Otherwise, if the algorithm reports that #F > N, we know 
with certainty that this is the case. Hence, given that the algorithm reports the latter, the algorithm of 

Lemma [3~4l will take time O* I -= — J to yield an e e -approximation of #F. 

It remains to bound the running time which amounts to optimizing the cutoff parameter N. For 
every choice of N, the combined algorithm works in time within a sub-exponential factor of 

™ 
max{2 l3kn N^ 1 "^\— }. 

Let / be such that log 2 N = f ■ n. Then this maximum translates into max{/3fc + /(l — /3jt), 1 — /}. 
Since (/3k + /(l — /?&) is increasing and 1 — / is decreasing in /, the minimum over all / of the maximum 
of the two is obtained when / is chosen so as to make them equal, that is, at 

1 2-/V 
This translates into an overall running time of 



O* {2 2 -f>k 



Recall that /?& determines the running time 0*(2™ n ) of the subroutine consisting of a randomized k- 
SAT algorithm, used in the algorithm of Lemma 13.21 We shall have a look at these running times, 
now. 

The case k > 5. The algorithm of Paturi, Pudlak, Saks and Zane IPPSZ05I . can be used as the 
subroutine randomized fe-SAT algorithm, which has a running time of 

O* (o^~^i)A _ (2) 

Hence, we have here, (3k = 1 — -S^i which yields the claimed bound. 



The cases k = 3 and k = 4. For these values of k, several improvements over the PPSZ algorithm 
have been presented. The currently fastest one is that by Hertli BHerlll . whose bounds match those 
of the PPSZ algorithm in the unique-SAT case. We thus also have here the corresponding bound of 
equation ©. 

4 Upper bounds 

In this section we will prove Theorem ll.3l bv presenting a simple algorithm producing upper bounds on 
#F. We will use Valiant and Vazirani's bisection technique HVV861 and its application to approximate 
counting. We will therefore consider random GF(2)-linear systems of equations of the form Ax = b. 
For some m < n these consist of an m x n matrix A, an m dimensional vector b and a vector x 
representing the variables of F. The entries of A and b are chosen independently and uniformly at 
random from {0, 1}. As such systems give rise to a family of pairwise independent hash functions of 
the form h{x) = Ax — b, we will use the following well known 

Fact 4.1 (Hashing Lemma). Let F be a CNF formula, and Ax = b an m x n random linear system of 
equations. Then 

Pr [\{x € sat(F) | Ax = b}\ £ (1 - e, 1 + e) • 2~ m • #F] < l — ■ ± . 

The proof of this lemma is straightforward and can be found e.g. in Goldreich's book IG0IO8II . 
Secondly, we need a standard fact about the rank of random matrices such as the matrices obtained in 
the above way. Consider a random m x n matrix A as above with m < n and let r denote its rank. 
The proof of the following lemma then follows easily, for example, from a result of Blomer, Karp and 
Welzl IBKW971 : 

Lemma 4.2. There is a constant c such that E//„[r] > m — c. Furthermore, r > m — O(logm) with 
probability at least 1 — 0{m~ l ). 

A third ingredient of the algorithm is the following Lemma, which is easily proved. 

Lemma 4.3. Let Ax = b be a system of GF(2)-linear equations with solution set A. There is an 
algorithm listing all solutions in time within a polynomial factor of\A\. 

We are now ready to prove the Theorem. 

Proof of Theorem [731 We start with the description of the algorithm. Choose a random GF(2)-linear 
n x n system of equations Ax = b. Starting with a parameter v = n and decreasing v in each step, 
we build random linear systems A n x = b n , A n _\x = & n -i> • • • > A^x = b^. The system A v x = b u is 
obtained from Ax = b by deleting the last n — v rows of A and entries of b. Then F v denotes the pair 
consisting of F and A v x = b v . We say that F v is satisfiable if there is a solution x € sat(F) such that 

AyX = V . 

For each v we rigorously determine whether F v is satisfiable by using the algorithm of Lemma [4731 
to list all solutions of the linear system and for each determine whether it satisfies F. We let u be the 
minimum v > p, such that F v is unsatisfiable. If all F n , . . . , F^ are unsatisfiable, we set u = p. 

To establish the time bound, note that the running time is dominated by the time the algorithm spends 
in determining satisfiability of Fp. By Lemma l4~2l the matrix A^ has with high probability rank at least 
m — 0(log n) and we have \A\ = 0*(2 n ~^) which yields the claimed time bound. Thus we shall in the 
following condition on the event that the rank of A^ satisfies this rank criterion. 



Correctness. The correctness follows from standard arguments also used in the classical approach BVV861 . 
We give a proof for completeness. Let / = [log #F~\ . Assume first, for simplicity, that /j, = 0. We will 
show that with the desired probability, it is a 16-approximation to #F. 

First, consider the event that u = f — c for some c > 4. The probability p c of this event is the 
probability that all of F n , . . . , F u are unsatisfiable. Furthermore, conditional on F u being unsatisfiable, 
all F v for v > u are unsatisfiable, as well. Hence, we have p c = Pt[F u is unsat ], and by the Hashing 
Lemma |4~T1 we have thus 

Pt[F u is unsat ] < — < 2^ c . 

By a union bound argument, we thus see that / — 3 < u with probability at least 3/4. 

Next, consider u = f + c which implies that F u _i is satisfiable. Applying the Hashing Lemma |4~T1 
with parameter e = £ — 1 for £ = 2 U ~ 1 /#F, we see that 

Pr[F M „i is satisfiable] < 



(C-i) 2 ' 

As C > 1, this bound is decreasing in £, and hence in u, therefore, the probability that F u _i is satisfiable 

is at most 2 C ~ 1 (2 C_1 — 1)~ 2 . Again, a union bound shows that u < / + 3 with probability at least 33/49. 

Next, note that if /j, > then these findings do not change. Especially, if fj, < f — 3 then the above 

result does not change. And if \x > / — 3, then by the above, with probability at least 3/4 we have 

u = fJL. 

Taking into account that we have conditioned on A^ having rank m — O(logm) the above prob- 
abilities degenerate a bit. But the bounds claimed in the statement of the Theorem are still easily 
achieved. □ 



Remarks. Note that the use of listing algorithm of Lemma 14.31 can be avoided by using a uniform 
sampling algorithm for the solutions of Ax = b, this then yields essentially the same time bounds. 
Furthermore, uniform sampling is easily achieved by fixing a basis of the column space of A, choosing 
u.a.r. assignments to non-basis variables and extending these assignments to solutions of Ax = b. 

5 Open Problems 

It is a peculiar fact that our result falls short of yielding any reasonable time bound for the approximation 
of #2-SAT. A direct application of the algorithm of Theorem 13.11 to this case would yield (using a 
polynomial time 2-SAT subroutine) a bound of 0(1.4142 n ) whereas the fastest exact method ITWah081 
takes time only 0(1.2377 n ). It would therefore be interesting to develop an approximation algorithm 
which beats the bounds of these exact methods. 

Secondly, the time bounds achieved in this paper are significantly better than those for known exact 
methods, but also, they are much worse than the bounds known for the corresponding satisfiability 
problems. Is it possible to close this gap, maybe even in terms of a purely algorithmic analog of Valiant 
and Vazirani's result? 
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